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Getting assessment results used is no easy task. This 
paper examines the findings of a recent study on the uses of National 
Assessment of Educational Progress (NAEP) and suggests ™Pj;£«tions 
for increasing use of state and local assessment results. The NAEP 
study conducted at Northwestern University, generated evidence that 
assessment results and methods were employed to make decisions, 
persuade others, and enhance understanding of issues. These uses 
occurred in the context of professional activities (the use of 
results to improve instruction); research (the use of results to 
understand achievement); and policy development (the use of results 
--bv-^TffeTeirrievels of governance) . State and local agencies often 
exploited NAEP findings and methods to assess student achievement and 
to identify and address weaknesses in curricula* programs, but there 
was a wide variety of other uses as well. Implications are discussed 
for assessment planning, meeting heterogeneous user needs, inventing 
multiple dissemination strategies, providing technical assistance, 
developing inter-institutional relations, and systematic monitoring 
of use. ( Author /PLB) 
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Our recent study of the utilization of the National' Assessment of 
Educational Progress (Sebring and Boruch, 1982) identified three broad 
categories of use: prof essional uses , -policy uses, and research uses. 
Professional use refers to the employment of NAEP data, methods, and materials 
to ultimately assess and improve educational programs and classroom in- 
struction, Most of the uses by state agencies , local schools, and curriculum 
organizations fall in this category. Policy use occurs when NAEP data are 
exploited to inform decision makers such as members of Congress, state 
legislators, or state and federal agency officials who regulate the use of 
funds. Research uses refer to the use of NAEP data to experiment with new 
measurement techniques or to understand the relationship between educational 
attainment and certain student and school background variables , These broad 
categories overlap somewhat. Use of data by professionals can lead to 

advocating certain policies about curriculum emphasis, and researchers 1 
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findings can have implications for -instruction and for policy. 

'professional use by the states mostly involves replicating or adapting 
the tfAEP model to conduct parallel assessments at the state level, which in 
turri lead to recommendations for curriculum development or revision. Twelve 
states have replicated and 14 states have adapted the NAEP model. Five of 
the former and two of the latter were included in this study. In general, 
these states involved educators in selecting objectives and items for the 
assessment and interpreting the findings. NAEP data for national and 
regional samples often functioned as norms against which the state could 
measure attainments of its students. Inmost of the states studied, assess- 
ment reports are distributed to local schools, and workshops are conducted 
to share findings and recommendations. In two states studied formal arrange- 
ments are struck with local schools to allow them to conduct their own 
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assessments using National Assessment and state items. Usually states reveal 
a history of assessment styles, and National Assessment has been an important 
contributor to this history. 

Local districts employ National Assessment and their state assessment 
results in a wide variety of ways. Sometimes the objectives are used as 
a resource in developing curriculum objectives; in other cases NAEP data 
items, or methods assist with decisions concerning choice of curriculum 
materials, teacher in-service programs, or development of diagnostic or 
competency tests. 

The national Council of Teachers of Mathematics (NCTM) and the National 
Council for Teachers of English (NCTE) fostered other prof essional uses. 
These organizations are two among many that contribute to NAEP policy, use 
data products of NAEP, and assist in interpretation and dissemination of NAEP 
results. 1 NAEP findings also served as a basis for NCTM's recommendations for 
mathematics curricula for the 1980' s. Furthermore, a wide variety of journals 
and magazines, both national and regional, covered NAEP results, and NAEP 
data repeatedly appear in books on the teaching of reading and writing. 

In terms of policy use, NAEP data have been presented to Congressional 
committees, including the hearings on the reauthorization of Title I in 
1978 and hearings on the Omnibus Reconciliation Act of 1981. NAEP has 
documented cooperation with several federal agencies to provide information 
pertinent to their programs, and more recently NAEP has begun developing 
policy papers pertinent to issues before Congress, such as aid to private 
education and Title I. In addition, case studies of states revealed that 
state legislatures have employed state assessment and National Assessment 
data to monitor the condition of education in their state as well as to 
consider legislation on competency testing. 
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Until 1979, research use of raw data generated by NAEP was low but 
not negligible. Transfer of NAEP to NIE led to dramatically increased 
research use, partly because of NIE T s mission and partly because of spe- 
cialized support for research from NIE and NSF. As a result of the latter, 
170 data tapes have been distributed since 1979 for analyses aimed at dis- 
covering better methods of measuring and reporting achievement and under- 
standing the relationships between learning and certain student and school 
background variables. 

NAEP data, methods, and materials were employed to make specific de- 

i 

cisions, and to persuade others or confirm beliefs. The most common use, 
however, was for general understanding of issues and problems". Utilization 
was found to be piecemeal, incremental, and fragmented. * It can best be 
understood as the gradual process of accumulating knowledge overtime and 
translating that knowledge into program or policy initiatives. 

In conclusion, the evidence available on use is sufficient to contradict 
-earlier claims that NAEP is not used. The uses can be documented at each 
level of government, .and by professional organizations, for assessment of 
children's achievement, curriculum planning, problem identification, and 
other purposes . ; 

Lessons for Local and State Education Agencies 
Some of the recommendations from the NAEP utilization study are rele- 
vant to state and local assessment activities. Below we have orClined 
seven "lessons" for increasing the use of assessment results. 

1. The potential user should be represented in the initial planning 
of the assessment . This is generally regarded as accepted practice and; 
our study simply confirmed the importance of this step. 
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In all the case studies where NAEP items and/or procedures were used, 
teachers and other professional staff participated in the planning, the 
selection of objectives and items, and sometimes in the interpretation of 
results. It is a recognized fact that early involvement of users will 
heighten chances of use later on. . 

2. The fact that user needs are heterogenous shoul d be recognized and 
accommodated . An important determinant in the widespread use of NAEP 
materials is their flexibility. For most content areas it is possible to 
obtain summary reports, objectives, test items and guidelines for con- 
ducting their own assessment, reprints of articles, interpretive reports, 
or djata tapes. This flexibility allows the user to select only that product 
whifch most specifically meets his or her need. 

likewise, a state or local agency should consider the diverse needs 
of the audiences it serves. For example, it may be worthwhile to produce 
several different types of reports. Teachers may want detailed analyses 
that permit identification of problems in the curriculum or the classroom. 
Policymakers, such as a school board, legislators, or the state board of 
education, may desire a more general report that highlights findings, shows 
relevant comparisons, and suggests policy implications. Administrators 
may need yet another type of report. Beside providing printed reports, 
states could offer technical assistance to schools who wish to conduct their 
own assessments. 

3. , SEAs and LEAs should establish a. policy of encouraging col laboration 
between their assessment activities a nd other studies. Special studies 
could be adjoined to a regular assessment, or the option of augmenting the 
sample for certain schools or school districts should be available. 
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For example, a state or local agency may want to determine the cor- 
respondence between assessment items and content covered in classrooms. 
There may be policy issues which could be assisted by adding a set of items 
or questions to the assessment test. Such arrangements enhance the use- 
fulness of the assessment by capitalizing on the existing sample, methods, 
and general structure . ' " 

In, states where assessment offices have limited resources to promote 
such cooperation, consortium arrangements between LEAs may be possible. In 
fact, special studies might be conducted by anyone with the resources to 
do a good job, such as professional organizations, administrators, school 
boards, or academic institutions. 

4. Multiple dissemination strategies are needed to reach and inform 
diverse audiences . One^etfiod is to disseminate information and promote 
utilization among groups and organizations that have links to relevant 
audiences. States often have counterparts to such national organizations 
as the National Education Association, American Educational Research Associ- 
ation, curriculum organizations such as the National Council for Teachers of 
English, etc. Furthermore, local bargaining agents, organizations of school 
district research and evaluation offices and regional educational labor- 
atories might be helpful in disseminating information. 

Institutionalizing relationships with groups can be accomplished by 
naming a representative in the organization to keep abreast of assessment 
activities, regularly reporting to statewide or local newsletters and 
journals, and arranging poster sessions for statewide or regional meetings. 
Furthermore, once findings are released, there could be a formal plan for 
briefing representatives of some of the groups listed above, as well as 
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legislators who serve on education committees, the press, the state board 
of education, state officers responsible for Title I Evaluation and Re- 
porting, and so on. 

To go beyond printed reports and magnetic tape files, assessment data 
could be disseminated through public television broadcasts and through the 
use of microprocessors in government agencies or in the larger' school dis- 
tricts. At both the state and local level there should be a system of 
collecting, cataloguing, and retrieving assessment reports. This would 
facilitate easy access to data on previous assessments, performance iti 
certain content areas, or of specific grade levels. 

5. Cheap systems for monitoring actual use of^ assessment dat a need to be 
developed and exploited . Such a system is critical for as s essing the costs 
and benefits of regular assessments . Our case studies and interviews re- 
vealed that most states have little or no evidence regarding use of state- 
wide assessment reports. The same may be true of local districts. 

It is important to obtain both quantitative and qualitative data in 
this area in order to understand the relative frequency of use, to move 
beyond reliance on case studies, and to make judgments about the utility of 
assessments when considered against alternatives. / 

A variety of options can be considered. First, it is conceivable that 
an inexpensive system of telephone surveys could be set up at the state 
level or local level to periodically survey the SEAs ' and LEAs" uses of 
the information. Such a system could rely on cheap expert labor, e,g. 
graduate students, and can reasonably be expected to result in an annual 
report on use. Periodicity of such surveys is important because of the 
influence of memory lapse on reports of use. 

/ 
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Where telephone surveys are not appropriate, mail surveys of a 
probability sample of LEAs might be. They are less likely to be satis- 
factory because obtaining reliable information about use requires some 
probing and guidance. The word use evokes different images for respondents, 
and memory failure and self interest may play a role in responses. Routine 
administrative reports on use by all SEAs and LEAs does not seem sensible 
since probability samples would be accurate. Moreover, requiring a 
system imposes further burdens on respondents, and our experience with 
administrative reporting on use in other sectors suggests the approach is 
not promising and can lead to gratuitous reports. 

. . Another vehicle is to include prepaid post-cards in each report along 
with a request to return them with information on use. This is unlikely to 
lead to an unbiased sample. In addition, small side studies would have to 
be run to assess quality rather than quantity of use. Some use is likely 
to be inept and other use remarkable. Cooperative arrangements between 
states or between local schools seems desirable because variations in 
practice can illuminate what works and why. 

6. Monitoring systems should define use explicitly in terms of 
audience, type of use, function of use, and the elements of the as sessment 
that are used . How we view use of assessment findings depends on our 
definition of use. Moreover, decisions at the state or local level about 
continued participation in assessments, investment of resources for augmented 
samples or special studies cannot be made without a reasonably clear idea 
of what the word implies. Finally statistical studies of the scope and 
frequency of use, and costs and benefits cannot be made without better 
definition. 



We suggest that in policy statements, evaluations of assessments, and 
in legislation, that use be defined in terms of 

. audience, i.e. state legislators, state and local agency officials, 
educational organizations, school boards, teachers and other 
local school staff; 

function of use, i.e., identify problems, make decisions, increase 
understanding, and persuasion; 
. elements of use: test items, objectives, sampling methods,- 

summary reports, technical assistance, etc; 
. and type of use, e.g. professional, policy, and research. 
These categories are applicable as well to 'gauging use of other kinds of 
information at the state and local levels, including use of special pro- 
gram evaluations and use of administrative reporting systems such as the 
Title I Evaluation and Reporting System. 

7. Finally, it is important to recognize the constraints on the use 
of information . Even when multiple strategies for dissemination are em- 
ployed, spread of information remains imperfect. Furthermore, even when 
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state agencies and schools gain a better understanding of student needs 
through assessment reports, they may not have the resources to implement 
changes suggested by the assessment. Decisions do not turn on information 
alone. Assessment reports and results may be only one element among many 
that are considered and can easily be outweighed by bureaucratic, budgetary, 
and political factors. 
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